System and method for testing of web services

ABSTRACT

The present invention is a module testing tool for Web services. In one embodiment, the present invention automates the testing of Web services that use SOAP as a wire protocol and HTTP as a transport protocol. The invention provides an easy interface for exercising Web services and testing their functionality. The invention helps users confirm the responses to SOAP messages with such features as fault detection, textual comparisons, XML validation by DTDs or XML Schemas, and the ability to express and flag complex patterns in XML. The invention lets the users validate responses that require application-specific verification (such as business logic validation) by plugging in their own code. It also provides the capability to perform regression testing of web services. The invention can automatically creates regression test controls from SOAP Client responses, or users can create their own regression tests.

CROSS-REFERENCE TO RELATED APPLICATION

This patent application is a continuation of U.S. patent applicationSer. No. 10/214,209, filed Aug. 7, 2002, which claims the benefit of thefiling date of U.S. Provisional Patent Application Ser. No. 60/312,010,filed Aug. 13, 2001 and entitled “MODULE TESTER FOR WEB SERVICES”, theentire content of which is hereby expressly incorporated by reference.

FIELD OF THE INVENTION

The present invention relates to a method and system for testingcomputer software. More specifically, the present invention is directedto a method and system for module testing of Web Service protocols.

BACKGROUND OF THE INVENTION

The problem of writing error-free computer programs has plaguedprogrammers since the very beginning. Sophisticated schemes forautomatically discovering program errors and bugs of all kinds,including lexical, syntactic, semantic and logical, have been developed.However, these schemes have generally been directed to enforcingadherence to concrete programming standards inherent in the definitionof the programming language itself and not to more extrinsicquality-related concerns, such as language modularity, portability,testing costs and communications between different program modules innetworked and distributed environments.

Web service is a term that has emerged to describe a software moduledeployed on the Web and intended for use as a component of one or moreapplications distributed across the Internet. A Web service inherentlyexists as part of a larger system. It is the servicing of these othercomponents that gives Web services their name. Thus, a primaryconsideration in the design of a Web service is the protocols that willbe used to communicate with other components of the system.

The first relevant protocol is the transport layer, which is typicallyHTTP or HTTPS. Other transport protocols, such as Java's Remote MethodInvocation (RMI), CORBA's Internet Inter-Orb Protocol (IIPO), or DCOM'sDistributed Computing Environment Remote Procedure Call (DCE RPC) canenable distributed applications, but such applications are notconsidered Web services because they are not deployed on the Web.

A second relevant protocol is the packaging layer. This layer defineshow content is wrapped up into messages that are sent over the transportlayer. The emerging standard in this area is called SOAP. SOAP,originally stood for “Simple Object Access Protocol”, but it now refersto a more generalized protocol for any service oriented protocol. SOAPis currently an XML-based messaging framework for exchanging structuredand typed information between peers in a decentralized, distributedenvironment. Extensible Markup Language (XML) is a meta-markup languagefor describing data objects. The specifications defining both SOAP andXML are produced by the World Wide Web Consortium (W3C). SOAP istypically implemented in XML and relies on XML namespaces and XMLschemas to define document types that describe messages. SOAP describesa simple messaging (request/response) mechanism (for example, RemoteProcedure Calls (RPC)). That is, a SOAP client sends a request (e.g., aRPC request) to a SOAP server. The SOAP server then replies by sending aresponse (e.g., a RPC response) to the SOAP client.

Web Services Description Language (WSDL), also typically implemented inXML, is used to describe the types of requests accepted by a particularWeb service. It is used to communicate meta-information about the Webservice and, as such, is not strictly necessary for invoking the Webservice itself.

Since XML is a meta-markup language, it provides mechanisms for defininglanguages that are implemented in XML. There are two basic mechanismsfor defining these languages: Document Type Definition (DTD) and XMLSchema. DTDs provide mechanisms for expressing which elements areallowed and what the composition of each element can be. Allowableattributes can be defined per element type, and allowable attributevalues can be defined per attribute. However, DTDs have someshortcomings. They do not provide the capability to extend types, theydo not have provisions for namespaces, and competency in DTD writingrequires learning a syntax that seems obscure to those outside the SGMLworld. To address these issues, XML schemas were introduced. Like DTDs,schemas define a set of authorized elements, attributes, and attributevalues. However, schemas are namespace-aware and are more type-orientedthan DTDs. Also, schemas themselves are an XML-based language, solearning schemas does not require learning a new syntax.

The development of high quality web services requires testing atmultiple levels. This is a difficult and time consuming task, both toexercise the web services and to confirm that the messages conform torequirements regarding features such as well-formedness, validity, faulthandling, and application-specific business logic. Therefore, there is aneed to a system and method to automate testing of SOAP-based Webservices.

SUMMARY OF THE INVENTION

Accordingly, the present invention enables the above problems to beovercome by providing a method and system for automating the testing ofWeb services that use SOAP as a wire protocol. The invention provides aneasy interface for exercising Web services and testing theirfunctionality. The invention helps users confirm the responses to SOAPmessages with such features as fault detection, textual comparisons, XMLvalidation by DTDs or XML Schemas, and the ability to express and flagcomplex patterns in XML. The invention lets the users validate responsesthat require application-specific verification (such as business logicvalidation) by plugging in their own code. The invention also providesthe capability to perform regression testing of web services. In oneembodiment, the invention automatically creates regression test controlsfrom SOAP responses. Alternatively, users can create their ownregression tests. In one embodiment, for testing XML, the inventionincludes a built-in Extensible Style Sheet Language Transformation(XSLT) processor, XML parser, and an editor.

In one aspect the invention is directed to a method for automatictesting of a web service software including SOAP comprising: identifyingthe web service; creating one or more test cases for exercising behaviorof the identified web service; executing the created one or more testcases for obtaining results for the behavior of the identified webservice; and analyzing the results. In another aspect, the invention isdirected to a method for automatic testing of a SOAP client comprising:identifying a web service interface to the client; creating behavior ofthe identified interface; exercising the client to invoke the webservice and for obtaining results for the behavior of the identifiedinterface; and analyzing the results.

Still other embodiments of the present invention will become readilyapparent to those skilled in the art from the following detaileddescription, wherein is shown and described only embodiments of theinvention by way of illustration of the best modes contemplated forcarrying out the invention. As will be realized, the invention iscapable of other and different embodiments and its several details arecapable of modification in various obvious respects, all withoutdeparting from the spirit and scope of the present invention.Accordingly, the drawings and detailed description are to be regarded asillustrative in nature and not as restrictive.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A is an exemplary simplified flow diagram of a testing process,according to one embodiment of the present invention;

FIG. 1B is an exemplary simplified flow diagram of a testing process,according to one embodiment of the present invention in which theinvention behaves as a SOAP server;

FIG. 2 is an exemplary graphical user interface (GUI) screen, accordingto one embodiment of the present invention;

FIG. 3 is an exemplary GUI screen for specifying test parameters,according to one embodiment of the present invention;

FIG. 4 is an exemplary GUI screen for specifying one or more output fora selected tool parameters, according to one embodiment of the presentinvention;

FIG. 5 is an exemplary GUI screen for specifying test parameters for aselected tool parameters, according to one embodiment of the presentinvention;

FIGS. 6A-6C are exemplary GUI screens for executing the tests anddisplaying the results, according to one embodiment of the presentinvention;

FIGS. 7A-7C are exemplary GUI screens for different error views,according to one embodiment of the present invention;

FIGS. 8A-8H are exemplary GUI screens for different settings, accordingto one embodiment of the present invention;

FIG. 9 is an exemplary GUI screen for an Editor tool, according to oneembodiment of the present invention; and

FIGS. 10A-10D are exemplary GUI screens for adding new projects andconfiguring HTTP connections, respectively, according to one embodimentof the present invention.

DETAILED DESCRIPTION

The present invention is a module testing tool for Web services. In oneembodiment, the present invention automates the testing of Web servicesthat use SOAP as a wire protocol and a transport protocol such asStandard Mail Transfer Protocol (SMTP), Hypertext Transfer Protocol(HTTP), HTTP over Secure Socket Layer (HTTPS), and the like. Theinvention provides an easy interface for exercising Web services,performing black box testing, and testing SOAP clients. In addition, theinvention can be used to confirm the responses to SOAP messages withfeatures such as fault detection, textual comparisons, XML validation byDTDs or XML schemas. Responses requiring application-specificverification such as business logic validation are supported throughscripting. The invention also performs regression testing of Webservices, including automatic creation of regression test controls fromSOAP responses. In the case of XML, one embodiment of the inventionincludes a built-in XSLT processor, XML parser, and editor.

FIG. 1A is an exemplary simplified flow diagram of a testing process,according to one embodiment of the present invention. In step 102, theWeb service to be tested is identified. For example, a WSDL file, aUniversal Description Discover and Integration (UDDI) file, or someother metadata about the Web service describing, for instance, whatprotocols are being used and how to interact with the Web service, maybe identified in this step. One or more test cases that exercise thebehavior of the identified Web service are created, as depicted in step104. In one embodiment, this step includes identifying methods andparameter types associated with the Web service, generating instancesfor the identified methods and parameter types, asserting propertiesabout the response based on metadata in the Web service, and storing theabove steps as a reusable test case. The created test cases are executedto obtain results for behavior of the Web service in step 106. In step108, the results are analyzed, for example, conformance to thespecification is validated.

In one embodiment, the invention is capable of automatic creation of GUIelements to input custom, complex data types. This means that when acustom, complex data type is specified in a WSDL document, the inventiongenerates controls for the specific elements which make up the data typein order to allow the user to easily specify complex parameters. Theinvention creates an internal representation of the metadata for the Webservice, which includes information such as the service name, number ofparameters, parameter types, and return type, which is incorporated intothe generated test case. From this internal representation, theinvention produces a graphical user interface representation thatvisually represents a test case. This visual representation of the testcase enables users to both analyze and manipulate the test casesdirectly. The parameter instance generation and manipulation are notlimited to the GUI, but the presence of the GUI facilitates this work inthe cases where it is most helpful. This is particularly true when verycomplex types are used and users may want to directly express specificinstances of the complex parameters so as to exercise known portions ofthe Web service implementation.

The GUI representation is a composite of complex and primitivecomponents. The primitive components are based on a mapping of primitiveXML types to primitive GUI components. For example, strings arerepresented with text boxes, boolean values are represented withcheck-boxes, arrays are represented with tables, enumerations arerepresented with combo boxes. The primitive XML types are mapped toprimitive GUI components. This mapping enables arbitrarily complex datatypes to be represented in the GUI by decomposing the data types intotheir constituent primitive types, translating these constituent partsinto their corresponding GUI components, and composing a complex GUIcomponent that parallels the composition of the original data types. Anexemplary process is depicted below:

-   -   a) Break complex data types into primitive data types including        the relationships of how the primitive data types are composed        into the complex data type,    -   b) Replace the primitive data types with their corresponding        primitive GUI elements based on a predefined mapping, and    -   c) Compose the GUI elements into a composite GUI element based        on the relationships found in the original complex type        composition.

In addition to working as a client to test Web services, the inventioncan be used as a server to test clients. When the invention is used inthis way, it serves as a platform on which Web services are deployed. Agood reason for deploying a service in the invention is that the sameverification tools used to test SOAP responses can be used to verifySOAP requests.

As an example, consider the manipulation of items in a shopping cart inan Online Grocer application. Normally, the shopping cart would bemanipulated via the Web interface. However, manipulating the applicationprogrammatically might also be desirable. For instance, say that theOnline Grocer forges a partnership with another online service and thatother service wants to purchase items for their customers. This is abusiness opportunity for the Online Grocer to gain some extra purchasesfrom its partner's customers. It is also a business opportunity for thepartner if the Online Grocer provides a commission for each purchasethat comes through this means. However, this type of tight integrationrequires that the partner be able to place orders programmatically.Services required could include:

-   -   addItemToCart( ): Adds a specified item to a specified cart    -   disposeCart( ): Deletes a specified cart    -   getCartItems( ): Returns the items in a specified cart    -   getCartTotal( ): Returns the total cost for a specified cart    -   getCartsByUserId( ): Returns all carts for a specified user    -   getNewCart( ): Creates a new cart for a specified user    -   purchaseCart( ): Purchases a specified cart

Deploying these services in this embodiment of the invention is simple.First, we choose View>Show Web Services. A Web Services tab appears inthe left window of the GUI. We select the Web Services root node withinthe Web Services tab and set the port number to the port on which wewant to publish these services. Now, we can add services and definemethods for SOAP RPC calls.

For example, to add the getCartTotal ( ) service, we right-click themain Web Services node and select Add Output>Web Service. A node labeledundefined method is added to the Web Services tree. After we select thenew service by clicking it, a method parameters panel appears in theright frame of the GUI. Here we can determine how the inventionunmarshalls parameters and marshalls return values. If we selectAutomatically, the invention will unmarshall parameters and marshallreturn values for us. In this case, the function signature shouldexactly match the RPC method that will be requested. If we selectManually, we ourselves can unmarshall the parameters and marshall thereturn values. We choose the automatic marshalling option here forsimplicity. Methods in the invention may be implemented in Java, Python,or JavaScript. A simple implementation of the getCartTotal( ) method inPython looks like:

def getCartTotal(cart_id):

-   -   return 12345

The input parameter cart_id is ignored, and the constant total of$123.45 is hardcoded.

In one embodiment, the invention is capable of Web Service deployment inmultiple languages including: Java, JavaScript, Jython (Java-basedPython), Jacl, Jscheme, and the like. These multiple languages includenative services as well. In this case, the invention behaves as a serverin order to test the client. In this embodiment, a variety ofimplementation languages are supported, where the user specifies bothwhich implementation language they are using as well as what actualimplementation to use, which is specified in their language of choice.Alternatively, the user can specify the implementation language, butallow the invention to generate an implementation that conforms to aspecified interface. These methods are not mutually exclusive because animplementation that is automatically generated may then be modified bythe user. Examples of implementation languages that the user may specifyinclude Java, Javascript, Python, and the like.

FIG. 1B is an exemplary simplified flow diagram of a testing process,according to one embodiment of the present invention in which theinvention behaves as a SOAP server in order to test a SOAP client. Instep 120, the Web service interface to the client is identified. Similarto step 102 of FIG. 1A, a WSDL file, a Universal Description Discoverand Integration (UDDI) file, or some other metadata about the Webservice describing, for example, what protocols are being used and howto interact with the Web service, may be identified in this step. Thebehavior of the identified interface is implemented in step 122. One wayto do this is to implement assertions about requests from the client.The client is then exercised to invoke the Web service in step 124. Inother words, the client is pointed to the created (predicted) behaviorto obtain results about the behavior of the client. In step 126, theresults are analyzed similar to step 108 of FIG. 1A.

In one embodiment, the invention includes the ability to check responsetime of Web Services. The invention is also capable of load testing forWeb services. It provides a flexible infrastructure that lets userscreate and run a wide variety of tests on their Web services. This canbe done in multiple ways. An assertion that the response time must bewithin a certain range can be expressed in the test case itself. It canalso be deferred to a post-execution analysis phase. In either case, atimer is used to determine the total time from the time the requestmessage is sent to the time the response message is received. Thisinformation can then be correlated with other data to disambiguatenetwork traffic time from server processing time. One such scheme is forthe invention to simply periodically ping the server to acquire anestimate of the network travel time. Another scheme is for the inventionto communicate with monitors on the server to deduce a finer grainedpicture of how much time is spent in each component. Another sub-itemhere is to record how the response time changes as the server is placedunder load, and to create a report of this for further analysis.

In one embodiment, the invention contains a library of tools that can beused as primary testing tools and/or as tools that perform operations onthe output of other tests. By combining and chaining the availabletools, complex and specific tests can be created.

The first step in testing a Web service with the present invention is tocreate a test suite. The test suite defines all the tests needed toperform on the service. A Graphical User Interface (GUI) includes a Testmenu containing the option Create Test Suite. Selecting this optioncauses a tree-view graphical representation of the new test suite toappear on the left side of the GUI. Right-clicking the test suite nodeprovides a menu for adding tests to the test suite. Each test has anassociated tool and an associated input. Examples of some availabletools include SOAP Client, XSLT, Validate XML, RuleEnforcer, and so on.

The following is a typical process for creating a test suite.

-   -   a) Create a new test suite by choosing Test> Create Test Suite.        A tab named New Test Suite will be added to the left side of the        GUI, as shown in FIG. 2. This is the tab where a new test suite        will be constructed and controlled.    -   b) (Optional) If you want to name your test suite something        other than “New Test Suite,” enter the new test suite name in        the Name field on the top of the right GUI panel.    -   c) Add the first test case to the test suite.        -   i) Right-click the root node of the Test Suite tree (the            node labeled Test Suite: <your test suite name>). A shortcut            menu will open, as depicted in FIG. 2.        -   ii) Choose Add Tool Test> <tool you want to use> from the            shortcut menu.        -   iii) Select the Test Suite tree node for the tool that you            just added, then specify the test parameters for that tool            in the right GUI panel, as shown in FIG. 3.    -   d) Specify one or more output for the tool just added.        -   i) Right-click the node that represents the tool to which            you want to add a new output (You can have multiple tools            perform operations on the result of one tool by adding            multiple outputs to the appropriate tool node.            Alternatively, you can have an output to perform an            operation on the result of another output by adding a new            output to an existing output node). A shortcut menu will            open, as shown in FIG. 4.        -   ii) Choose Add Output> <tool you want to use> from the            shortcut menu. Many basic output tools are listed under the            New Output menu. Tools that have already been used in the            current test are listed in the Existing Output menu.        -   iii) Select the Test Suite tree node for the output tool            that you just added, then specify the test parameters for            that tool in the right GUI panel, as shown in FIG. 5.        -   iv) Repeat steps i-iii if you want to add additional            outputs.    -   e) Add additional test cases by repeating steps c-d above. To        remove a test case, right-click the test case that you want to        remove, then choose Remove from the shortcut menu.

When the invention runs test cases, it performs all operations specifiedby selected tests and output parameters. A SOAP client test involvessending a request, receiving the response, and verifying the response byapplying other tools. Creating a SOAP client test requires selecting theSOAP Client tool and specifying the call information. The SOAP Clienttool behaves as a client in order to exercise the server. In oneembodiment, the invention includes automatic serializers, enabling asimple way to complete values in the GUI. Specify the URL to a WSDL toenable the invention to extract information such as the RPC Router, themethods contained in the service, and the target URIs and signatures ofeach method.

To run all test cases in the test suite:

-   -   Right-click the root node of your Test Suite tree, then choose        Run from the shortcut, as illustrated in FIG. 6A.

In one embodiment, the invention runs all available test cases, thenreport the outcome of each test (success or failure) and the testsuite's overall success rate, as depicted in FIG. 6B.

The results of a particular tool can be accessed by choosing that tool'sname in the View Selector (in the bottom right corner of the GUI) or byclicking the appropriate node of the Results tab in the left side of theGUI, as shown in FIG. 6C.

To run a single test case:

-   -   Right-click the root node of the test case that you want to run        (for example, the Test: SOAP RPC node), then choose Run from the        shortcut menu.

You can access the results of a particular tool by choosing that tool'sname in the View Selector (in the bottom right corner of the GUI) or byclicking the appropriate node of the Results tab in the left side of theGUI.

Regression test provides an easy way to determine whether anyfunctionality of the Web service changes. The invention canautomatically create regression controls for your SOAP Client and RuleEnforcer test cases. When prompted to do so, it creates Diff controlswith the current outcomes of the selected test cases, then adds theseDiff controls to the appropriate test cases. To automatically create aregression test control based on existing outputs, right-click the noderepresenting a single test or the node representing the entire testsuite, and choose Create Regression Control from the shortcut menu. Theinvention then adds a Diff tool to all selected tests. Each time thistool is used, it compares the results of the control test to the currenttest run.

The following is a typical way to have the invention automaticallycreate regression controls for the test suite:

-   -   Right-click the root node of your Test Suite tree, then choose        Create Regression Control from the shortcut menu.

To have the invention automatically create a regression control for aspecific test suite:

-   -   Right-click the Test Suite tree node that represents the test        case for which you want to create a regression control, then        choose Create Regression Control from the shortcut menu.

The next time the invention runs a test case with a Diff control, itwill compare the actual outcomes with the outcome specified in the Diffcontrols. If you ran the entire Test Suite, failed regression tests willbe reported in the main test results. You can always access the resultsof a particular regression test by choosing the appropriate Diff tool'sname in the View Selector (in the bottom right corner of the GUI) or byclicking the appropriate Diff node of the Results tab in the left sideof the GUI.

If a user wants to perform regression testing on a test case, but doesnot want to use the current outcome as the comparison value, preferablythe user needs to define her own Diff control for that test case.

To define a Diff control for a test case:

-   -   a) Right-click the root node of the test case that you want to        run (for example, the Test: SOAP RPC node), then choose Add        Output> New Output> Diff from the shortcut menu.    -   b) Select the Output: Diff node, then indicate what reference        value you want the invention to use. If you want to enter the        value, select the Text button, then enter the reference value in        the Text field. If you want the invention to use a value stored        in a file, select the File button, then enter or browse to the        file that contains the reference value.

The next time the invention runs this test case, it will compare theactual outcome with the outcome specified in the Diff control. If youran the entire Test Suite, failed regression tests will be reported inthe main test results. You can always access the results of a particularregression test by choosing the appropriate Diff tool's name in the ViewSelector (in the bottom right corner of the GUI) or by clicking theappropriate Diff node of the Results tab in the left side of the GUI.

If you want to repeat a test suite after you are done with testing, youshould save it so that you can easily re-open and re-run it.

To save a test suite:

-   -   a) Right-click the unused area of the Test Suite tab for the        test suite that you want to save. A shortcut menu will open.    -   b) Choose Save As from the shortcut menu. A file chooser will        open.    -   c) In the file chooser, specify where you want to save this test        (.tst) file.

The invention will then save the current test suite in the specifiedlocation.

To close a test suite:

-   -   a) Right-click the unused area of the Test Suite tab for the        test suite that you want to save. A shortcut menu will open.    -   b) Choose Close from the shortcut menu.

The invention will then close the current test suite and remove theappropriate tab from the left side of the GUI.

To open an existing test suite:

-   -   Choose Test> Open Test, then use the file chooser to specify        which test suite you want to open. A tab for the selected test        suite will be added to the left side of the GUI.

Table 1 includes some exemplary tools available for tests and/oroutputs. TABLE 1 Name Purpose Test? Output? Check Well Checks whetherXML files are well- Y Y Formedness formed. Create XML Creates adictionary from which you can Y Y Dictionary create your own rules. DiffCompares files or outcomes. Y Y Editor Sends results to internal editor.N Y External Invokes an external tool. Y N Tool File Writer Invokes theFile Writer tool that can Y Y create files. Java Method Sends output toa Java method. Y Y Rule Applies the rule or set of rules associated Y YEnforcer with that enforcer. SOAP Client Send SOAP messages. This toolis the Y Y foundation of most test cases Transmit Lets you transmit andreceive over a Y Y socket connection. XSLT Transforms XML files. Y YValidate Checks whether XML files are valid Y Y XML

You can use most of the above tools as tool tests. Some SOAP tests usethe SOAP RPC as the foundation of the test, then add other tools toprocess the results.

For example, if you wanted to test if a certain SOAP remote proceduralcall always returned the same output for a given input, you might createa SOAP Client test, then use a Diff control as its output.Alternatively, if you wanted to test whether a Web service returnedvalues in a correct format, you might create a SOAP Client test, thenuse a Rule Enforcer to apply a set of rules that checked if the outputmatched a certain required pattern. If it did, you could attach aregression control to the Rule Enforcer; the invention would then alertyou if the Web service failed to match that required pattern insubsequent tests.

Check Well-Formedness tool checks whether XML files are well-formed. Awell-formed document should preferably conform to the followingguidelines:

-   -   All elements that require both start and end tags have both        start and end tags.    -   Elements nest properly within each other.    -   Attribute values should be quoted.

This tool can be used as part of a test case, or it can be applied to afile or set of files opened in the Project tree.

Before you run this tool as a tool test in a test case, you need tospecify what the tool should check. (If you use this tool as an output,the invention assumes that you want it to check the result of the testor output to which it is attached). You can specify this in the CheckWell-Formedness Parameters panel. To open this panel, select the Test:Check Well-Formedness node in your Test Suite tree.

Create XML Dictionary tool creates a dictionary that you can use tocreate custom rules and queries. The dictionary created by this toolcontains a library of the elements used in the file from which it wascreated.

Before you run this tool as a tool test in a test case, you need tospecify what file or text the invention should base the dictionary on.(If you use this tool as an output, the invention assumes that you wantit to use the result of the test or output to which it is attached). Youcan specify this in the Create XML Dictionary Parameters panel. To openthis panel, select the Test: Create XML Dictionary node in your TestSuite tree.

Diff tool compares two files or outcomes and reports what (if any)differences there are between the two files or outcomes. If you areusing this tool as an output, you need to specify the reference valueagainst which you want to compare the input value. You can specify thisvalue in the Diff Parameters panel. To open this panel, select theOutput: Diff node in your Test Suite tree.

If you are using this tool as a tool test, you need to specify the inputvalue and the reference value against which you want to compare theinput value. You can specify both of these values in the Diff Parameterspanel. To open this panel, select the Test: Diff node in your Test Suitetree. In the Diff Parameters panel, specify the file or text you want tocheck in the Input portion of the panel, then specify the referencevalue in the Tool portion of the panel.

Editor tool sends data to internal editor. This tool is primarily usedto provide a way to access the files that the File Writer tool creates.To modify Editor settings, choose Editor> Preferences. This will open anEditor panel that you can use to modify the following options:

-   -   Tab Setting: Lets you determine the number of spaces inserted        after the tab button is pressed and whether or not the editor        keeps tabs.    -   End-Of-Line Marker: Lets you determine whether the editor uses a        UNIX or MS-DOS format end-of-line marker.

External tool let users include an external tool in their test cases.Before you run this tool as part of a test case, you need to provide theinvention with information about how to access and use it. You canspecify this in the External Tool panel. To open this panel, select theExternal Tool node in your Test Suite tree.

In the panel, you can specify the following parameters:

-   -   Name: Enter the name of this tool.    -   Executable: enter (or browse to) the name and path of the        executable file associated with this tool.    -   Arguments: Enter any arguments that you want to pass to that        tool.    -   Keep output: Select this option if you want this tool's output        to be displayed in the Results panel    -   Output pattern: Determines the output's format. Enter format        using regular expressions. For example, if a tool's output        pattern was 1:magic one.htm (where 1 is the line number,        magic_space is an expression, and one.htm is the file name), you        would enter: (.*)(:magic )(.*). This expression says that for        anything to the expression “:magic”, then anything. (anything is        represented by (.*)).    -   Pattern keys: Tells the invention the line number and file name        from the output pattern, in the terms of your selected source        editor. For example, if you were describing the output pattern        above (1:magic one.htm) in an editor where “l” represents line        number, “s” is used as a dummy symbol, and “F” represents file        name, your pattern keys would be lsF. If you were using Write,        your pattern key might be F.    -   MIME Types: Specifies the types of files the tool works with.    -   Input: Specifies the file or text to which you want the        invention to apply this tool.

File Writer tool can convert other output data to files. The Editor toolis often used in conjunction with this tool because it provides a meansto view the files that the File Writer creates. If you are using thistool as an output, you need to specify how you want this tool to nameand save files. You can specify these parameters in the File WriterParameters panel. To open this panel, select the Output: File Writernode in your Test Suite tree. You can then specify the followingparameters:

-   -   Name: Determines the name of this tool.    -   Target Name: Determines how this tool names the file it creates.        You can enter a specific name for a file, or use wildcards where        appropriate. Some acceptable wildcards include:        -   % b: Base name of original file (no extension).        -   % of: Original file name (with extension).        -   % e: Extension of original file name.    -   Target Directory: Determines where this tool places the file it        creates.    -   Override directory from input: Determines whether this tool        saves the files it creates in the same directory as the source        file (if available). If this is not checked and the Target        Directory specifies a directory other than the one that contains        the original source files, the FileWriter will save the file in        the same directory as the source file.

If you are using this tool as a tool test, you need to specify what datathe File Writer should write to a file as well as information about howyou want this tool to name and save files. You can specify theseparameters in the File Writer Parameters panel. To open this panel,select the Test: File Writer node in your Test Suite tree. You can thenspecify the parameters as described above.

Java Method tool lets you send output to a public static Java method.This is useful if you want to perform application-specific checks thatcannot be expressed with a rule (for example, if you want to check if agiven output matches a record in a database). The method used shouldhave a single parameter of type ‘Object’ and a return type of ‘boolean’.It should return “true” for passed tests and return “false” for failedtests. To use this tool, you need to specify the name of the classcontaining this method and the name of this method. You can specify thisin the Java Method Parameters panel. To open this panel, select theOutput: Java Method node in your Test Suite tree.

Rule Enforcer tool applies the customized rules that you have createdand associated with a particular enforcer. The Rule Enforcer's actionsare determined by the nature of the associated rule(s). Rules can lookfor problems in the data returned by your Web service, check whether itsresponses always match certain general patterns, or simply query yourWeb service. You can check whether a Rule Enforcer produces consistentresults by attaching a regression control or Diff output to that RuleEnforcer.

If you are using this tool as an output, you need to specify what ruleor rule set it should enforce. You can specify both of these values inthe Rule Enforcer Parameters panel. To open this panel, select theOutput: Rule Enforcer Parameters node in your Test Suite tree.

If you are using this tool as a tool test, you need to specify what ruleor rule set it should enforce as well as what file or text it shouldapply this rule to. You can specify both of these parameters in the RuleEnforcer Parameters panel. To open this panel, select the Test: RuleEnforcer node in your Test Suite tree. In the Rule Enforcer Parameterspanel, specify the file or text you want to check in the Input portionof the panel, then specify the rule or rule set in the reference valuein the Tool portion of the panel.

SOAP Client tool is used to perform SOAP messaging. This tool is thefoundation of most test cases. To perform a SOAP Client test, you needto tell the invention how to execute the call. This involves specifyingthe following parameters in the SOAP RPC Parameters panel (this can beaccessed by selecting the Test: SOAP RPC node in your Test Suite tree):

-   -   WSDL URI: Enter the WSDL URI where this Web service can be        accessed.    -   Description: (Automatically completed if available): Describes        the Web service at the given WSDL URI.    -   RPC Router: (Automatically completed if available) Describes how        to access the RPC router.    -   Method name: (Automatically completed if available) Lists the        available methods you can call and test. The list of methods is        created automatically when a valid WSDL URI is entered. You can        select which method you want to test by selecting a method from        the list.    -   Return type: (Automatically completed if available) Lists the        selected method's return type.    -   Target object URI: (Automatically completed if available) Lists        the target object's URI.    -   Encoding Style URI: (Automatically completed if available) Lists        the encoding style URI used to send requests.    -   Parameters: (Automatically completed if available) Lists the        parameters available for the selected method.    -   Name: (Automatically completed if available) Lists the name of        the selected parameter.    -   Type: (Automatically completed if available) Lists the type of        the selected parameter.    -   Encoding: (Automatically completed if available) Lists the        available encoding URIs for receiving requests. This list is        created automatically. You can select which encoding type you        want to use by selecting a type from the list.    -   Value: Determines if you want to use a literal value (i.e., a        string) or a generated value (i.e., if you want to        programmatically specify the value to pass as a parameter).    -   Value text field (below the value option): Lets you specify the        string or generated value that you want to pass a parameter. If        you are using a generated value, you need to enter a static Java        method that returns an object and that meets the signature the        Web service is expecting. The invention will then transform the        class into a SOAP parameter and send it as a SOAP call when the        test case is executed.

After you run a test, the output of the test will be displayed in theOutput window at the bottom of this panel. If the MIME type of theoutput is not text/plain, you need to select the proper MIME type valuefrom the drop-down list in order to have your output reported correctly.

Transmit tool lets you transmit and receive over a socket connection. Itis used for communicating directly with a specific port of a host. Itcan also be used to listen at a particular port if you are expectingsomething sent to some specific port on your machine.

You can use this tool to perform a lower level of testing than SOAPClient allows because it looks at the socket connection, which ishandled by HTTP and sits one layer below the SOAP call. It is alsouseful if you have applications communicating through XML over a socketconnection; it can be used to perform functional and regression testingof components that communicate using socket-based protocols. This toolcan be used as part of a test case, or it can be applied to a file orset of files opened in the Project tree.

If you are using this tool as an output, you need to specify your hostmachine and the port number that you want to use. You can specify thisinformation in the Transmit Parameters panel. To open this panel, selectthe Output: Transmit node in your Test Suite tree. If you are using thistool as a tool test, you need to specify what information it shouldtransmit as well as your host machine and the port number that you wantto use. You can specify this information in the Transmit Parameterspanel. To open this panel, select the Test: Transmit node in your TestSuite tree.

To listen using the transmit tool, right-click the Transmit Test Suitetree node, then choose Start Listening from the shortcut menu thatopens. Typically, you do not need a host specified in the Transmitparameters panel if you want to listen; the host parameter is onlyrequired if you want to transmit. To stop listening, right-click theTransmit Test Suite tree node, then choose Stop Listening from theshortcut menu that opens. To close the connection to the host,right-click the Transmit Test Suite tree node, then choose CloseConnection from the shortcut menu that opens.

Transmit tool supports some output options. These options include:

-   -   stdout    -   stderr    -   Result Window    -   Packetize by New Lines    -   Packetize by Documents    -   Packetize by MIME Messages    -   File

XSLT tool transforms XML files using the style described in thespecified XSL file. This tool can be used as part of a test case, or itcan be applied to a file or set of files opened in the Project tree. Ifyou are using this tool as an output, you need to specify what XSL fileit should use to transform the XML it receives. You can specify this inthe XSLT Parameters panel. To open this panel, select the Output: XSLTnode in your Test Suite tree. In this panel, you can also specifywhether you want this tool to store information in using relative paths(instead of absolute paths) and the MIME type of the transformeddocument.

If you are using this tool as a tool test, you need to specify what XMLyou want to transform as well as what XSL file it should use totransform the specified XML. You can specify both of these parameters inthe XSLT Parameters panel. To open this panel, select the Test: XSLTnode in your Test Suite tree. In the XSLT Parameters panel, specify thefile or text you want to transform in the Input portion of the panel,then specify the XSL files that you want to use in the Tool portion ofthe panel. In the Tool portion of the tab, you can also specify whetheryou want this tool to store information in using relative paths (insteadof absolute paths) and the MIME type of the transformed document.

An Editor output is added to this tool by default. Preferably, theFileWriter is used as an output so that the invention writes the outputto files.

Validate XML tool checks whether XML files are valid. An invalid XMLfile does not contain a proper document type declaration and/or does notobey the constraints of that declaration (correct elements, nesting,attributes, attribute values, etc.). This tool can be used as part of atest case, or it can be applied to a file or set of files opened in theProject tree. Before you run this tool as a tool test in a test case,you need to specify what file the invention should check. (If you usethis tool as an output, the invention assumes that you want it to usethe result of the test or output to which it is attached). You canspecify this in the Validate XML Parameters panel. To open this panel,select the Test: Validate XML node in your Test Suite tree.

Test results are displayed in the right side of the GUI. If you testedwith a single tool, any problems found will be displayed in the Resultspanel (in the right side of the GUI). If you ran a test suite or singletest case, you can access individual error messages by either:

-   -   Choosing a specific test (for example, Rule Enforcer or the name        of a test suite) using the View Selector (in the bottom right        side of the status bar).    -   Clicking the appropriate node in the Results tab (in the left        side of the GUI).

Some of the tools (such as Rule Enforcers, Validate XML, and CheckWell-Formedness) report output in tree or table format. When results arereported in this format, the following options are available:

-   -   Browse the file associated with the error message: To do this,        right-click the error message, then choose Browse from the        shortcut menu.    -   Edit the file associated with the error message: To do this,        right-click the error message, then choose Edit from the        shortcut menu.    -   Edit the rule related to a Rule Enforcer error message: To do        this, right-click the error message, then choose Edit Rule from        the shortcut menu.    -   Suppress an error message: To do this, right-click the error        message, then choose Suppress from the shortcut menu.    -   Clear all results in the current results category (e.g., Rule        Enforcer): To do this, right-click the error message, then        choose Clear from the shortcut menu.    -   E-mail the results: To do this, right-click the error message,        then choose Mailto from the shortcut menu.    -   Save the results: To do this, right-click the error message,        then choose Save As> <desired format> from the shortcut menu.

Additional options may be available; the options that are availabledepend on the specific to the type of error message you are viewing, aswell as the format in which you are viewing results. If your results aredisplayed in tree format, you can quickly expand the entire Results treeby right-clicking the unused area of the Results panel, then choosingExpand All from the shortcut menu.

You can change the way that the invention displays results in theResults panel to suit your personal preferences and the nature of yourcurrent task. Typical options include:

-   -   Table, by file    -   Table, by error    -   Tree, by file    -   Tree, by error

Typically, table vs. tree tends to be a personal preference: trees aremore structured than tables, but tables consume less window space.Whether you want to see your results listed by file or by locationusually depends on what task you are trying to accomplish.

To change the way that results are displayed:

-   -   a) Right-click anywhere in the Results panel. A shortcut menu        will open.    -   b) Choose View> <desired view type> from the shortcut menu.

Results will then be displayed in the format that you selected. Theinvention will continue displaying errors in the selected manner untilyou configure it to do otherwise. FIG. 7A shows an exemplary screen fora tree view by error. FIG. 7B depicts an exemplary screen for a treeview by file. FIG. 7C illustrates an exemplary screen for a table view.

You can suppress error messages that you do not want to see using thesuppression feature of the invention. For example, you may want tosuppress a particular error as soon as you have corrected it; this willmake it easier for you to see what errors still need to be corrected.You could also suppress errors that are not relevant to your project.

When you suppress an error message, you are given a choice betweensuppressing the error message permanently or temporarily. A message thatis suppressed permanently will be suppressed for all future test runs,until you remove the permanent suppression option. Temporarysuppressions will automatically be removed the next time that you runthe tool that produced the error.

To suppress a particular error message:

-   -   a) In the Results panel, right-click the error message that you        want to suppress. A shortcut menu will open.    -   b) Choose Suppress from the shortcut menu.    -   c) In the window that appears, click Just this error, then click        the appropriate button to indicate whether you want the        invention to suppress this error temporarily or permanently.    -   d) Click OK.

To suppress all errors that occur in a file:

-   -   a) Right-click any error that occurs in the page whose errors        you want to suppress.    -   b) Choose Suppress from the shortcut menu.    -   c) In the window that appears, click All errors in <filename>,        then click the appropriate button to indicate whether you want        the invention to suppress this error temporarily or permanently.    -   d) Click OK.

You can also suppress all errors that are caused by the same file or“rule.” For example, you could suppress all errors related to (but notin) junk.htm, a file that you are going to remove from your site, or youcould suppress errors relating to a specific type of test or rule.

To suppress all errors that have the same cause as a selected error:

-   -   a) Right-click any error that has the same cause (a file or a        rule) as the type of errors that you want to suppress.    -   b) Click Suppress in the shortcut menu.    -   c) In the window that appears, click All errors from <source of        error> or All references to this page, then click the        appropriate button to indicate whether you want the invention to        suppress this error temporarily or permanently.    -   d) Click OK.

If you later decide that you want to see results that you havepermanently suppressed:

-   -   a) Open the Edit Suppressions window by doing one of the        following:        -   Right-clicking any error message, then choosing Edit            Suppressions from the shortcut menu that opens.        -   Choosing View> View Suppressions.    -   b) In the Suppressions window, select the suppression item you        want to unsuppress.    -   c) Click Delete.

This suppression will then be removed. Errors that were previouslysuppressed by this suppression will be reported in future test runs. Youcan use the suppression counter to keep track of how many times theinvention has encountered a specific error or type of error that yousuppressed permanently. To view the number of hits for each suppression,choose View> View Suppressions, then look at the number of hitsdisplayed next to each suppression. If you want to reset the number ofhits for a certain suppression to zero, select the suppression whosehits you want to reset, then click Reset Hits.

When you add a suppression, it is applied only to files in the site fromwhich you created the suppression. Such suppressions are calledsite-specific suppressions. If you want certain permanent suppressionsto apply to all sites that you test in the invention, you should changethe suppression to a user-specific suppression. To change a suppressionfrom site-specific to user-specific (or vice versa):

-   -   a) Open the Edit Suppressions window by doing one of the        following:        -   Right-clicking any error message and choosing Edit            Suppressions from the shortcut menu that opens.        -   Choosing View> View Suppressions.    -   b) Select the suppression item whose type you want to change.    -   c) Click Swap. The suppression item will then be moved to the        alternative suppression type (from site-specific to        user-specific, or from user-specific to site-specific).

You can customize many preferences in the Preferences panel. To accessthis panel, choose File> Customize Preferences. Once this panel is open,you can modify the following settings.

You can customize settings related to the tools in the invention toolbar by modifying options in the Preferences panel's Tools tab, as shownin FIG. 8B. You can modify existing tools' settings by clicking the nodethat represents the tool you want to configure, then changing theoptions available in the right side of the panel. You can also add anexisting tool (such as Create Dictionary or Diff) or an external tool tothe tool bar by clicking the new button, then selecting the tool thatyou want to add from the dialog box that opens, as depicted in FIG. 8A.

In one embodiment, the invention is configured to work with anyRCS-based source control: if you have your source control set upcorrectly, you can check files in and out of your source code repositoryby right-clicking the Project tree node that represents the file thatyou want to check in or check out, then choosing the appropriate Checkin or Check out option from the shortcut menu that opens.

To integrate your source control system:

-   -   a) Choose File> Customize Preferences to open the Preferences        panel of FIG. 8B.    -   b) In the Preferences panel's RCS tab, check the Enable RCS        check box, as shown in FIG. 8C.    -   c) In the Check Out Command field, enter the name (and full        path, if necessary) of your check out (“co”) command.    -   d) In the Check In Command field, enter the name (and full path,        if necessary) of your check in (“ci”) command.    -   e) In the RCS field, enter the name (and full path, if        necessary) of your “rcs” command.    -   f) In the File Extension field, enter the file extension of        files under software configuration management. This value is        usually, v but some ports have been configured to use other        extensions (for example, % v).    -   g) (Optional) If you use RCS to store the archived files in a        subdirectory, enter that name in the Directory field. Typically,        the name is “RCS”.

When you browse Web files, the invention will open them in your system'sdefault browser unless you customize the browser settings in thePreferences panel, as shown in FIG. 8D. The Browser panel includes thefollowing browser options:

-   -   Browser: Determines what browser the invention uses.    -   Command: Determines browser commands such as executable name and        arguments that you want the invention to pass to that browser.        If you select a browser name that is provided, the invention        will automatically fill in both Executable and Arguments. If you        select Other, you can click Browse to navigate to the        appropriate Executable settings. For Arguments, enter “% 1”.    -   Use DDE: Determines whether or not Dynamic Data Exchange (DDE)        lets programs share information. If you select Use DDE, the        changes you make to files as you are using the invention will        automatically be applied to other programs that use those same        files. Use DDE is selected by default and may not be disabled        for the Automatic option in the Browser field. It is selected by        default but may be disabled for Netscape Navigator and Internet        Explorer. When Other is selected in the Browser field, Use DDE        is disabled by default and may not be enabled.    -   Click OK.

The invention works with proxy servers. To configure it to work withyour proxy server, customize the settings in the Preferences panel'sProxy tab illustrated in FIG. 8E.

-   -   If you want to use the current system proxy configuration, check        the System Proxy Configuration check box. The invention will        then enter your system proxy settings in the grayed out fields.    -   If you want to specify proxy settings, make sure the System        Proxy Configuration check box is clear, then enter your proxy        setting information in this tab.    -   If you want to use the same proxy server for all protocols,        check the Same proxy server for all protocols check box, then        enter the address and port of the proxy server you want to use        in the Proxy Address and Proxy Port fields.    -   If you want to use different proxy servers for different        protocols, clear the Same proxy server for all protocols check        box, then enter the address and port of each proxy server you        want to use in the Proxy Address and Proxy Port fields.    -   If your proxy server requires authentication, check the Enable        proxy authentication check box, then enter a valid user name and        password in the Username and Password fields.

The exemplary MIME Types tab of FIG. 8F lets you add and remove MIMEtypes. In addition, it lets you specify the location of your preferredtext and XML editors and lets you specify what editor you want to use toedit all files that have a particular MIME type.

You can map public XML identifiers to system XML identifiers in thePreferences panel's XML Identifiers tab, as shown in FIG. 8G. To createa new mapping, click Add, then use the Browse button to select both yourpublic and system identifiers. To modify an existing identifier, selectit, then use the Browse button to select a new identifier. To remove anexisting identifier, select it, then click Remove.

By default, the invention saves the WSDL URIs that are used in createdtest cases. The list of WSDL URIs used is contained in the Preferencespanel's WSDL tab. If you do not want the invention to save these WSDLURIs, clear this panel's Add from Test Suite check box, as shown in FIG.8E.

Additionally, the invention includes an internal editor you can use toview and edit files. An exemplary Editor screen is illustrated in FIG.9. To open an existing file in the editor, choose Editor> Open, thenchoose the file that you want to open from the file chooser that opens.To create a new file in the editor, choose Editor> New File, then startcreating your file. To save a file, choose Editor> Save or Editor> SaveAs.

Other conventional editor commands (such as revert, copy, paste, etc.)are also available in the Editor menu and from the shortcut menu thatcan be opened by right-clicking the any area of the Editor panel. Tocustomize the editor, choose Editor> Preferences. You can then modifytab and end-of-line marker settings in the Editor dialog box that opens.You can also access the menu options from the Editor menu byright-clicking the Editor panel.

If you want to test or transform a set of files (such as a group offiles in a certain directory or the files that are on a Web server) youmight want to create a project that contains all of your files. You canthen apply the Validate XML, Check Well-Formedness, Rule Enforcer,Transmit or XSLT tool to all files, files in a certain directory, or asingle file.

A preferred way to work with a set of files is to create a project. Aproject is a file that can contain one or more site files. Each sitecontains information about how to access a source. For example, if youare working on two different XML-based Web applications, you might wantto create one project file that contains two site files (one for eachapplication you are working on).

Project files provide you with a central place to access and workdifferent groups of files. To create a project file:

-   -   a) Choose Project> New Project. The Project tab will open and        contain a node named New Project.    -   b) (Optional) Assign a name to the project by right-clicking the        New Project node, choosing Properties from the shortcut menu,        then entering the project's name in the Name field of the        Project Properties panel that opens.

You can add multiple sets of files (sites) to a project file. To add aset of files to the Project:

-   -   a) Right-click the root node in your Project tree. A shortcut        menu will open, as shown in FIG. 10A.    -   b) From the shortcut menu, choose Add New Site. The Add New Site        panel will open.    -   c) Indicate the source of the files you want to transform or        test in the Source tab. This tab contains the following fields:        -   Web Server: Enter the name of your Web server (for example,            http://www.parasoft.com/index.htm). You can configure            additional options by clicking the icon next to the Web            Server field. As shown in FIG. 10B, available options            include:            -   Default Pages: Enter the name of your default pages                (e.g., index.html, index.htm, default.htm, etc.). To                speed up loading, enter an asterisk (*) in this field.            -   CGI Directory: Enter the directory where your CGIs are                located.            -   Log Directory: Enter the directory where your server log                is located.            -   Obey Robot Restrictions: Tells the invention whether or                not to obey robot restrictions. Because robots are often                restricted from accessing programs, you should clear                this option when you are loading dynamic web sites. You                should only clear this option to access sites where you                have the permission of the Web site administrator.                Typically, this should only be used to get easy access                to your own pages without having to modify your Web                server.            -   Pass Cookies: Tells the invention to pass cookies from                the site being scanned to the hard drive on which the                invention is being run. This option should be checked                when you are loading dynamic sites; if it not, you may                not be testing the site as thoroughly as possible.            -   Invoke CGI for All Arguments: Tells the invention to                process links to CGI scripts like: <A                HREF=“/cgi-bin/script.cgi?TEST=1”> This option is                recommended if you are loading a dynamic site; if it is                not enabled, the invention may not be able to reach your                dynamic pages.            -   Scan Forms with Default Input: Tells the invention to                automatically submit each form it encounters with the                default values (the equivalent of the user simply                clicking Enter or Submit). This option is recommended if                you are loading a dynamic site.            -   Fill Active Inputs Manually: Tells the invention to let                you manually enter values for each form that it                encounters. You should select this option when you are                loading dynamic sites in which submitting default values                will not fully exercise your site.            -   Case Insensitive Server: Tells the invention that the                named Web server is not case sensitive.            -   FTP: Enter the name of the FTP server you use to access                your files, then configure additional options by                clicking the button next to the FTP field, as depicted                in FIG. 10C. Configuration options include:            -   Host: Enter the name of the FTP host.            -   Username: Enter your username used to access this FTP                server.            -   Password: Enter your password for this site. Note that                if you choose to store your password here, it may be                available to anyone who has access to your project or                site file where the password is stored. If this file is                not secure, you should leave this field empty. If you                do, you will be prompted for your password whenever it                is needed.            -   Directory: Enter the full path to the directory where                your files are stored.        -   Local Path: Enter the name of the path where your files are            locally accessible. You can choose a path by clicking the            button next to the Local Path field.        -   Telnet: Enter the name of the Telnet server you use to            perform operations on your files. You can configure            additional options by clicking the button next to the Telnet            field. Available options include:            -   Host: Enter the name of the Telnet host.            -   Port: Enter the port number.            -   Username: Enter your username.            -   Password: Enter your password for this host.            -   Directory: Enter the directory where your files are                stored.            -   Prompt: Enter any arguments that you want automatically                entered at the prompt. For example, if your prompt looks                like (username)→, enter (username)→here.    -   d) Click OK. The files will then be added to your Project tree.

If want to add files from other directories/sources to the current site,you will need to add them using indirect links. You should only useindirect links to add a small set of files related the current site; ifthe files that you want to add are not closely related to the files inthe current site, you might want to add an additional site for thosefiles.

To use an indirect link to source files to a site:

-   -   a) In the Project tree, right-click the directory under which        you want the file or directory to be located. A shortcut menu        will open, as shown in FIG. 10D.    -   b) Select Create Indirect Link from the shortcut menu. The Add        Indirect Link dialog box will open.    -   c) Complete the Add Indirect Link dialog box.        -   i) Indicate whether you are adding a directory or a single            file by selecting the appropriate button.        -   ii) Enter the name you want to assign to this link.        -   iii) Indicate where this source is currently located. Local            path and FTP are the preferred type of source, but it best            to enter as many media as possible.        -   iv) Click OK to close this box and add the specified file or            directory to your Site tree.    -   d) Repeat the above steps to add any additional indirect links.    -   e) Choose Project> Refresh to refresh your site.

The files/directories referenced in the indirect links will then beadded to your Project tree. To save the current project and site files,choose Project> Save.

When you open a project file, all associated site files will also beopened. To open a previously saved project files and the site files itcontains, choose Project> Open Site/Project, then choose the .wkj filethat represents your project. Before you can test or transform files ina project, preferably, you should have a project open.

To apply Validate XML, Check Well-Formedness, or any Rule Enforcer toolto all project files, a certain directory, or a certain file:

-   -   a) Select the Project tree node that represents the files that        you want to test. To test all files in the project, select the        project's root node. To test all files in a site, select the        site's root node. To test all files in a directory, select the        directory's root node. To test a single file, select that file's        node.    -   b) Click the tool bar button that represents the tool that you        want to apply.    -   c) If you want the invention to test all directories or paths        recursively, click OK in the dialog box that opens. If you only        want to test the selected directory or path, clear the check        box, then click OK.

The invention will then apply the selected tool and report errors foundin the right side of the GUI.

Before you apply the Transmit tool to transmit files in your Projecttree, you need to configure it. To do this:

-   -   a) Right-click the button for the Transmit tool that is designed        to transmit files (this is the button that has an empty        receiver), then choose Configure Transmit from the shortcut        menu. The Configure Transmit dialog window will open.    -   b) In the Configure Transmit window, enter your host name and        specify which port you wan to use.    -   c) If you want the invention to keep the specified connection        after transmitting the specified data, select the Maintain        connection check box.    -   d) Click OK to close this window and save your settings.

To transmit a single file or all files in (and below) and certaindirectory, select the Project tree node associated with the file or setof files you want to transmit, then click the appropriate Transmitbutton. You can also “listen” to socket connections by clicking theappropriate Transmit button. Before you listen, right-click the Transmittool bar button with the busy receiver, choose Configure Transmit fromthe shortcut menu, then specify the port you want to use in theConfigure Transmit window that opens.

To apply the XSLT tool to all project files, a certain directory, or acertain file:

-   -   a) Select the Project tree node that represents the files that        you want to transform. To transform all files in the project,        select the project's root node. To transform all files in a        site, select the site's root node. To transform all files in a        directory, select the directory's root node. To transform a        single file, select that file's node.    -   b) Click the XSLT tool bar button.    -   c) In the dialog box that opens, specify the .xsl file that you        want the invention to use to transform the first file. If you        want to use the same .xsl file to transform all files in the        selected directory, select the remember this file option.

The invention will then transform the indicated file(s).

If you want to configure the XSLT tool bar button to always use acertain XSL file or if you want your transformed documents to have aMIME type other than text/xml, right-click the XSLT tool bar button,choose Configure XSLT from the shortcut menu, then change the parametersin the Configure XSL window that opens.

When the invention finds an XML file that does not contains a properdocument type declaration and/or does not obey the constraints of thatdeclaration (correct elements, nesting, attributes, attribute values,etc.), it reports an XML Validation error. The reason for rule isbecause XML files that are not valid might not be parsed and processedcorrectly.

For example, the following file is not valid because the NAME element isnot declared in the DTD. <?xml version=“1.0”?> <HR> <NAME id =“Engineering”>John Smith</NAME> <NAME id = “Marketing”>Jane Smith</NAME></HR> <NAME id = “Marketing”>Jeremy Young</NAME>

The file and/or the DTD is then corrected so that the file references aDTD and conforms to that DTD.

When the invention finds an XML document that does not follow the syntaxof XML (i.e., a document that is not “well-formed”), it reports an XMLNot Well-Formed error.

A Well-formed document typically conforms to the following guidelines.

-   -   All elements that require both start and end tags have both        start and end tags.    -   Elements nest properly within each other.    -   Attribute values should be quoted.

The reason for rule is because XML files that are not well-formed mightnot be parsed and processed correctly.

It will be recognized by those skilled in the art that variousmodifications may be made to the illustrated and other embodiments ofthe invention described above, without departing from the broadinventive scope thereof. It will be understood therefore that theinvention is not limited to the particular embodiments or arrangementsdisclosed, but is rather intended to cover any changes, adaptations ormodifications which are within the scope of the appended claims. Forexample, SOAP originally stood for Simple Object Access Protocol, but isnow merely a name. In fact, there is a movement to expand the meaning ofthe acronym to “Service Oriented Architecture Protocol.” Therefore, theterm SOAP, as used in the specification and the claims, is not limitedto Simple Object Access Protocol, but it covers any service orientedprotocol. Furthermore, the claimed invention is not limited to XMLimplementations. Other implementations using similar languages also fallwithin the scope of the invention.

1. A method for testing of a web service software including serviceoriented architecture protocol (SOAP) and metadata, the methodcomprising: creating one or more test cases for exercising behavior ofthe web service; creating an internal representation of the metadata forthe web service; incorporating the internal representation of themetadata into the generated test case; and executing the created one ormore test cases for obtaining results for the behavior of the identifiedweb service.
 2. The method of claim 1, further comprising identifyingcomprises identifying a Web Services Description Language (WSDL) relatedto the web service.
 3. The method of claim 1, further comprisingidentifying comprises identifying a metadata related to the web service.4. The method of claim 1, further comprising modifying the created testcases for future use.
 5. The method of claim 1, further comprisingvalidating conformance to specification included in the SOAP.
 6. Themethod of claim 1, further comprising confirming responses to the webservice.
 7. The method of claim 6, further comprising enforcing codingrule against the responses to the web service.
 8. The method of claim 1,wherein the SOAP is implemented in Extensible Markup Language (XML). 9.The method of claim 8, further comprising validating the XML.
 10. Themethod of claim 8, further comprising identifying complex patterns inthe XML.
 11. The method of claim 1, further comprising determining theresponse time of the web service.
 12. The method of claim 11, whereinthe web service is executed on a server, the method further comprisingdetermining how the response time of the web service changes as theserver is placed under load.
 12. The method of claim 1, furthercomprising load testing of the web service.
 13. The method of claim 1,further comprising chaining a plurality of testing tools to createcomplex tests.
 14. The method of claim 1, further comprising comparing areceived response of a test case to an expected response of the testcase.
 15. The method of claim 1, wherein the metadata for the webservice includes one or more of a service name, a number of parameters,a parameter type, and a return type.
 16. The method of claim 1, furthercomprising generating a graphical user interface (GUI) representation ofthe test case.
 17. The method of claim 16, further comprising displayingcomplex components and primitive components in the GUI, wherein theprimitive components are based on a mapping of primitive XML types toprimitive GUI components.
 18. The method of claim 1, further comprisingregression testing of the web service. 19 The method of claim 1, furthercomprising creating rules to check whether a response of the web servicematches certain general pattern, and enforcing the rules.
 20. A computersystem for testing of a web service software including service orientedarchitecture protocol (SOAP) and metadata, comprising: means forcreating one or more test cases for exercising behavior of the webservice; means for creating an internal representation of the metadatafor the web service; means for incorporating the internal representationof the metadata into the generated test case; and means for executingthe created one or more test cases for obtaining results for thebehavior of the identified web service.